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Distortion in transmission channels causes inter symbol interference in 
digital communication systems. This distortion may be partially corrected 
at the receiver through the use of a tapped delay line having adjustable tap 
gain settings (transversal filler) . The problem of minimizing distortion with 
a finite-length transversal filter is examined. In the region of small initial 
channel distortion where most existing systems operate, the best tap gain 
settings satisfy a set of simultaneous linear equations. For larger initial 
distortion, iterative techniques are required to find best gain settings. The 
distortion is shown to be a convex function of the tap gains, so mathematical 
programming techniques may be employed for optimization. 

The practical problem is that of evolving a logical strategy whereby the 
tap gains of the transversal filler may be set to optimum values. An easily 
implemented system for automatic equalization is described which makes 
use of a steepest -descent technique of minimization. The equalizer is auto- 
matically set prior to data transmission in a training period during which 
a series of test pulses is transmitted. Only polarity information is re- 
quired, so digital logic may be used in the equalizer. For application to 
high-speed data transmission, great accuracies are required for the tap 
gain settings. Thus the problem of noise in the channel during equalization 
is quite important. The final error due to noise and channel distortion and 
the equalizer settling time are discussed and evaluated. Finally, the effect 
of a transversal filter equalizer in terms of the system frequency-domain 
characteristics is considered. 
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I. INTRODUCTION 

Present data rates on voice telephone channels are limited to about 
2400 bits per second. Although the noise margin on these facilities is 
sufficient tp permit much higher rates, the nonuniform transmission 
characteristics of the channel cause what might be termed a distortion 
barrier, prohibiting faster transmission. The distortion of data pulses by 
the channel results in these pulses being smeared out in time so as to 
overlap other transmitted pulses. This intersymbol interference is one 
of the chief degrading factors in present systems and becomes the de- 
termining factor in the design of higher-rate systems. To alleviate the 
effects of intersymbol interference it is necessary to equalize the channel. 

In the past equalization has generally been accomplished by flattening 
the amplitude characteristic and linearizing the phase characteristic 
using fixed amplitude-frequency and phase-frequency networks. Al- 
though this type of equalization is adequate for speech transmission 
requirements, it does not provide the precise control over the channel 
time response which is necessary for high-speed data transmission. Thus 
to realize the full transmission capability of the channel there is a need 
for automatic, time-domain equalization. 

Among the basic philosophies for automatic equalization of data 
systems are pre-equalization at the transmitter and post-channel 
equalization at the receiver. Since the former technique requires a feed- 
back channel, we will concentrate our efforts here on equalization at the 
receiver. This equalization can be performed either during a training 
period prior to data transmission or it can be performed continuously 
during data transmission. The typical voice channel changes little 
during an average data call, so pre-call equalization should be sufficient 
in most cases. Many of the principles and techniques which will be 
discussed here can be applied to the continuous or adaptive equalizer, 
although our main concern will be in the pre-call automatic equalizer. 

Most engineers are agreed that multilevel vestigial sideband trans- 
mission offers the best hope for higher-speed transmission on voice 
channels. There are good theoretical reasons for this choice of modem 
on a channel of limited bandwidth and high signal-to-noise ratio. The 
equalization problem we will discuss is based on the use of VSB trans- 
mission or what is equivalent, baseband transmission. The equalizer is 
to be placed at the receiver directly after the demodulation process. 
Thus as far as the equalizer is concerned the transmission is baseband. 

Suppose now that the equivalent baseband system transmits ampli- 
tude a„ at time nT, where a n is chosen from a set of M possible discrete 
amplitudes. The single-pulse response of the over-all system, including 
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channel and equalizer, we call h(l). The received signal y(t) is then 

y(l) = E aji(t - nT). (1) 

n=— oo 

At some suitably chosen sampling time / the output voltage y(t ) may 
be abbreviated 

oo 

y = E **- (2) 

H= — 00 

where 

fc„ = ftOoTwr). (3) 

This voltage is the sum of the wanted term a plus an intersymbol 
interference term 

yo = ho a + j- E Qnh-n • (4) 

[_ /lo n=-oo J 

(The prime will be used very frequently on summations to indicate 
deletion of the n = term.) 

Now in the second term of (4) the a n coefficients are chosen by the 
data system user according to some probabilistic rule. Since in a VSB 
system the maximum positive and negative of values of a n are equal 
in magnitude, say a max , the maximum value the intersymbol inter- 
ference term can assume is 

00 / 

max interference = — ^ /, \h n \. (5) 

ho n =-=o 

We thus define an interference criterion proportional to (5) called dis- 
tortion and labelled D 

D = r El*. I- (6) 

ho n — oo 

The so-called eye opening for an .V-level VSB system is simply and 
monotonically related to D. 

I = i - (.1/ - l)/). (7) 

The eye opening is a widely accepted criterion of a data system's per- 
formance. In what follows we shall always use the equalizer to mini- 
mize the distortion D defined by (6). A particular advantage of this 
criterion is that it is not dependent on noise statistics or on the statisti- 
cal distribution of the customer's data sequence {a n \. It is a minimax 
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criterion in that we seek to maximize the customer's minimum margin 
against noise over all data sequences. A heuristic argument can be 
made for a criterion using the sum of the squares of the h„'s. Obviously 
neither criterion minimizes average probability of error — a mathe- 
matically intractable problem even when assuming Gaussian noise and 
independent, equally likely input symbols. Practically speaking the 
minimization of either criterion leads to negligibly different results. 

Now what we need is a variable filter for an equalizer which can 
exercise wide control over the time response samples \h n }. The trans- 
versal filter shown in Fig. 1 is ideal for this purpose. 12 This filter con- 
sists of a continuous delay line tapped at T-second intervals. Each tap 
has an associated attenuator which in combination with an inverter is 
capable of giving variable gain. The filter output is the sum of all the 
attenuated tap voltages. Such a filter is capable of flexible control of the 
output time sequence {h„) when used in tandem with the channel to be 
equalized. In fact, under suitable conditions, derived in the final section 
of this paper, a transversal filter of infinite length may be used to com- 
pletely eliminate distortion in a channel. Generally speaking, a trans- 
versal filter of finite length cannot eliminate distortion, so we will 




Fig. 1 — Elements of a transversal filter. 
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study the minimization of distortion D with a transversal filter of 
finite length. We then show an implementation of an automatic equalizer 
which achieves minimum distortion under certain conditions generally 
satisfied on voice telephone channels. 

In succeeding sections the theoretical and practical limitations of the 
equalizer as implemented are studied. In order to effectively eliminate 
distortion in voice channels great accuracies are required in the setting 
of the tap gain coefficients. Background noise and settling time for the 
equalizer become extremely important. Finally we discuss the behavior 
of the transversal equalizer, essentially a time-domain device using a 
time-domain criterion, in the frequency domain. 

II. THE MINIMIZATION OF DISTORTION 

We will assume that the transversal filter has N + 1 taps with as- 
sociated tap gains. One of these taps is taken as the reference with its 
gain denoted c ( , , while the other N taps are placed somewhere at 
integer multiples of T seconds before or after tap c . The positions of 
these taps are usually the locations - (N/2)T, (- (N/2) + 1)7\ ■■■ , 
(N/2) 7', but since these particular locations are not necessary to any 
of the theorems we use a more general formulation. Let K N be the set 
of integers denoting the positions of the N + 1 tap locations 

K N = \n | tap location exists at time nT from reference}. 

The tap gains will be denoted c> , j C K N . 

The impulse response at the input to the transversal filter is denoted 
x(t) and its samples at times nT form the time sequence {.r„|. We will 
assume for convenience that this response is normalized so that x = 1. 
Thus the distortion in the pulse x(t) prior to equalization, called initial 
distortion D , is 

D = E |*>|. (8) 

The transversal filter serves as a device to multiply time sequences. 
In this case the input sequence |.r„j is multiplied by the tap gain se- 
quence \c„\ according to the rules of polynomial multiplication. It can 
be seen by inspection that the output sequence {h,,} is formed by the 
rule 

h„ = 2 CjZn-j ■ (9) 

The final distortion we seek to minimize is 
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D = i- E |A.|. (10) 

The reference response sample ho causes somewhat of a practical 
problem. In a multilevel system the slicing levels are generally fixed so 
that the gain must be closely controlled. In other words it is necessary 
to provide a normalizing control to adjust h to unity, the assumed 
value of Xq . This can be done two ways which lead to slightly different 
mathematical problems. 

Problem 1: 

Fix the tap c at unity. Minimize the criterion D over the TV variables 
[cj] ; j C K N , j r± 0. The output pulse may be normalized if desired by 
an over-all gain control outside the transversal filter. 

Problem 2: 

Let the tap Co be variable. Minimize the criterion D over the N + 1 
variables \cj\,j C K N , subject to the constraint h = 1. 

A study of these two problems reveals that the minimum distortion 
in either case is the same. Except in isolated cases (co = in Problem 2) 
the optimum tap gains in Problems 1 and 2 are related by a constant 
factor. There is a practical difference in the range of tap gains required 
and a mathematical difference in the type of function involved. (Prob- 
lem 1 is nonlinear, while 2 is piecewise linear.) Since the minima are the 
same and the optimum gains related, we concentrate on Problem 2 
because of its simpler properties. 

The constraint on h may be written 



= 1 = £ CjX-j. (11) 

1C.KN 



Solving for the gain c we have 



co = 1 - T/cjX-j. (12) 

l'CK N 



We substitute (12) into (9) to obtain 

K = H' Cj(Xn-j - .C„.r_;) + X„ . (13) 

Now since ho is unity the distortion (10) becomes 

D = E' | E' Cj(x„-j — X^C-j) + x n I . (14) 

n=-oo ICKN 
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Now that Co has been eliminated to satisfy the constraint on h we de- 
sire to minimize D in (14) over the N variables Cj ; j cz K N , j y* 0. 

First observe that D is a continuous, piecewise-linear function of the 
variables \cj\. We can rewrite (14) in the form 

CO =0 I 

D = ]£' Cj £' (.(•„_> - .r„.r_y) sgn /;„ + Y, &* sgn ft,, (15) 

/CA'.V B— « »— « 

sgn/ '" = \-l, ft.<0. 

In this equation the coefficients of the c/s are constant over certain 
regions of the A-dimensional space of definition of [cj\. Breakpoints 
where the coefficients assume new values occur whenever an output 
sample h„ is zero. A minimum cannot occur between breakpoints where 
the function is linear; thus at least one value h kl is zero at the minimum. 
The equation /?*, = may be used to eliminate one of the N variables 
d . The reduced equation is of the same piecewise linear form, requiring 
at least one more output sample h k „ = 0, etc. We arrive at the con- 
clusion that at least jV samples of the output time sequence {h n } must 
be zero at the minimum. But N equations of the form /i*. = 0; i = 1, 
■ • ■ , N are sufficient to determine the values of the tap gains {c,} . 
We need only solve JV simultaneous linear equations using (13) with 
n = /«•,• ; i = 1, • • • , N. 

The question remains as to the values of the A-/s, i.e., which iV zeros 
in the output response sequence does one force to achieve minimum 
distortion? In most cases of interest this question is answered by the 
following theorem, which is proved in Appendix A. 

Theorem I 

If I) < 1, then the minimum distortion D must occur for those N tap 
gains which simultaneously cause h„ = for all n CZ K N , n 7^ 0. 

Another important property of the distortion function which is both 
useful and descriptive is the following: 

Theorem II 

If the tap Co is used to satisfy the constraint h = 1, then the distortion 
D is a convex function of the .V variables Cj ;j C K N ;J5*0. 

Proof 

For convenience denote settings of the equalizer by the Af -component 
vectors a and a. To prove convexity of D it is necessary to show that 
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for any two settings a and a and for all X, ^ X ^ 1, 

D[Xa + (1 - X)<7] ^ X7)(a) + (1 - X)D(<r). (1(5) 

This equation would show that the distortion always lies on or be- 
neath a chord joining values of distortion in N -space. 
From (14) 

D[\a + (1 - X)d] = E' | E' IX«J + (1 - X)*/] 

»— /CK.v (17) 

• (.T„_; - .(•„.)•_;) + .r„ | 

/;[xa + (i - x)<ri = E' | x { E' «,(.(•„_> - .<•„.(•_,) + xn] 



>C*K 



+ (1 - X) { E'^C^n-y- .«•„.»•_,) + ;r„} 



(18) 



D[Xa + (1 - X)ff] ^ XD(a) + (1 - \)D(a). (19) 

One of the most important properties of convex functions is that they 
possess no relative minima other than their absolute minimum. Thus 
any minimum of D found by systematic search or other mathematical 
programming methods must be the absolute (or global) minimum of 
distortion. 

In summary, we have shown that D is a continuous, piecewise-linear, 
convex function of the N tap gains. This function has a single minimum 
which must occur when N zeros appear in the output time sequence 
\h„\. If the initial distortion is less than 100 per cent (Do < 1), then the 
minimum occurs when the N samples of the output time sequence which 
correspond in location to the N taps on the transversal filter are simul- 
taneously zero. This description is illustrated in two simple cases in 
Figs. 2 and 3. In Fig. 2 only one tap is variable (N = 1), while in Fig. 3 
equal distortion contours are plotted for an example of a 2-tap equalizer. 
In both cases the initial distortion is less than 100 per cent, so the minima 
are easily located. 

III. EQUALIZATION STRATEGIES AND IMPLEMENTATION 

3.1 Strategy When D u < 1 

The condition Do < 1, which is sufficient for easy location of the mini- 
mum distortion, is equivalent to the condition that the unequalized 
channel is capable of supporting binary transmission without error in 
the absence of noise. It may be seen from (7) that D = 1 implies a 



AUTOMATIC EQUALIZATION 



000 



1.5 
















1.0 






/ 










0.5 






























-0.5 0.5 

TAP SETTING, C, 



Fig. 2 — Distortion vs tap setting in a one-dimensional example. 




Fig. 3 — Equal distortion contours for a two-tap example. 
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completely closed binary eye. In almost all cases of interest on voice 
telephone channels the modem will support binary transmission without 
equalization, and so the condition is met. In these cases equalization 
enables multilevel operation to take advantage of a relatively high 
signal-to-noise ratio. For the present we will deal with the design of an 
equalizer where Do < 1. 

To make matters more concrete we will henceforth assume that the 
N adjustable taps of the transversal filter are divided equally before 
and after the reference tap. In the several equalizer models which have 
been built the normalization has been carried out sometimes by an 
outside gain control and sometimes by an adjustable center tap. In 
either case the task of the automatic tap gain setting apparatus is to 
zero the N output samples h n ; | n | ^ N/2, n j* 0. By Theorem I this 
achieves minimum distortion. One can, of course, derive the tap gains 
by the simultaneous solution of N linear equations, but from an instru- 
mentation point of view there are many simpler schemes. The condition 
Do < 1 ensures a "loosely coupled" system where the interaction be- 
tween tap gains is weak. Thus there are a number of iterative strategies 
which converge to the desired settings. In all these schemes a sequence 
of test pulses is transmitted prior to actual data transmission. After 
each test pulse the tap gains of the equalizer are readjusted in such a 
way as to eventually result in the proper N zeros in the output time 
sequence. The choice between such schemes is dictated by ease of in- 
strumentation, settling time required for equalization and accuracy of 
the final settings. 

It is true that with a special-purpose computer the optimum tap 
settings could be computed using only a single test pulse; however, this 
overlooks the presence of background and impulse noise on the facility. 
Each test pulse is in itself unreliable, so many pulses must be averaged 
in some way to give accurate settings. The equalization system must 
also be relatively unaffected by any large bursts or impulses which 
occur during the setup period. 

An equalization strategy has been devised which meets all require- 
ments and is easily instrumented. The motivation for this strategy is 
based on a steepest-descent technique. Consider the use of an outside 
gain control for normalization with the center tap gain fixed at unity. 
Solving the N simultaneous equations h n = for | n | ^ N/2, n ^ 
is equivalent to minimizing the "truncated distortion" 

D N = Y! I ^ |. (20) 
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The function D N is a convex function amenable to solution by steepest- 
descent techniques. After each test pulse has been received the tap 
gains are incremented so that the JV-dimensional incrementing vector 
is in a direction opposed to the gradient of D N . This gradient may be 
written 

Nl - An 
VD N = £' d -^aj (21) 

i— tr/i dCj 

where dj is a unit vector in the direction of the Cj coordinate. The com- 
ponents of the gradient are 

-=— = 2. — sgn/i„ (22) 

dCj n N/2 OCj 



dDs 



Nil 
OCj n—N/2 



Now we approximate (23) by assuming the samples x„ , for n 5^ 0, 
are small in comparison to x , which is unity. 



dD N 

dCj 



sgn hj (24) 



AT/2 

VD N £b 2' sgn kjdj . (25) 

v — N/2 

The approximation has resulted in an extremely simple expression, since 
all steps are of equal magnitude and the direction of each step is de- 
termined by simply taking the polarity of the corresponding output 
sample. No analog voltages are involved, so digital logic can be used 
in the tap gain setting circuitry. Before describing this circuitry, it is 
necessary to demonstrate that this iterative scheme does indeed con- 
verge to the desired minimum. After each test pulse, each tap gain 
Cj is incremented by an amount —A sgn h jt so that the new output 
samples are 

C = E' (c; - A sgn hj)x n -j + .r„ (26) 

I— N/2 

N/2 

h„* = h„ - A X' sgn hjXn-j (27) 

i—N/2 

N/2 

h n * = hn - A sgn h„ - A Y/ sgn hj.v,,^ (28) 

J=-N/2 
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\h„*\ ^ | | A. | -A | + A/A.. (29) 

Since Mi < 1, we have 

| A/ | < | | A, | - A| + A. (30) 

If | A* | > A, then 

|A»*| < | A. | (31) 

and each output sample that we desire to zero is always decreased. 
On the other hand, if | /*„ | < A, then 

| A.* | < 2A. (32) 

Thus the process must converge to within an error of 2A on each out- 
put sample. As the step size A goes to zero the truncated distortion 
D N approaches zero, which implies that the over-all distortion D has 
been minimized. 

3.2 Equalizer Implementation 

An equalizer implementing this strategy is shown in Fig. 4. This 
equalizer employs a 13-tap delay line with 6 variable taps on either 
side of the reference tap. The action of the equalizer is as follows: A 
succession of test pulses is sent through the transmission line and trans- 
versal filter. As each test pulse comes out of the transversal filter it is 
sliced to retain only polarity information and then sampled at T-second 
intervals. These polarity samples are stored in a 12-stagc shift register. 
When the shift register is full, a gate is opened, and all taps are simul- 
taneously adjusted one step up or down in attenuation according to 
the polarities in the shift register. At this time a pulse height or normali- 
zation adjustment is also made on the over-all gain by means of the 
upper slicer-sampler circuit shown in Fig. 4. The method of obtaining 
electronically controlled steps of attenuation for the equalizer taps is 
shown in Fig. 5. It can be seen that reversible counters, directed by the 
12-stage shift register, control the attenuation of a ladder network by 
means of relays. 

In the particular equalizer which has been constructed, the quantum 
steps on the tap gain controls are about 0.25 per cent (A = 0.0025). 
For high-quality voice channels utilized by the VSB data system, about 
100 test pulses are required for settling of the tap attenuators to opti- 
mum values. This settling time depends on the size of the quantum 
step, the initial distortion, and the channel noise. The settling time 
and residual distortion are the subject of the following section on 
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Fig. 4 — Automatic equalizer. 

equalizer performance. Note that the equalizer is relatively unaffected 
by large impulse noise, since the output voltage is sliced and a large 
impulse can cause only one wrong step, which is subsequently corrected. 
Figs. 6 and 7 show the results of equalization in a typical example 
using the 12-tap automatic equalizer. The single-pulse response together 
with the binary and 8-level eye patterns are shown before and after 
equalization. Before equalization only binary transmission was possible 
in this example. Since the equalizer has enabled 8-level operation in 
place of binary, a threefold increase in speed capability of the channel 
has been obtained. 

3.3 The Initial Distortion Limitation 

Thus far we have only considered equalization strategy when the 
initial distortion is less than 100 per cent. When I.\ > 1, there are two 
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Fig. 6 — Unequalized pulse and corresponding "eye patterns." 
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Fig. 7 — Equalized pulse and corresponding "eye patterns." 



sources of failure for this equalizer. First, the equalizer convergence 
algorithm may fail to converge to the tap gains which force zeros in 
the output response for / = - (N/2)T, ••• , (N/2)T. Second, these 
settings may not be the optimum (minimum distortion) gains. In 
either case the condition Do < 1 is sufficient but not necessary to prevent 
failure, and it is quite possible that the equalizer will converge and be 
optimum over a wider range of inputs. 

In computer simulations of the equalizer strategy it has thus far been 
impossible to induce failure of the convergence algorithm without 
causing failure in optimality. Examples of the converse are, however, 
easily constructed. One such example is illustrated in Figs. 8 and 9. 
The channel to be equalized in this example has a 50 per cent cosine 
roll-off amplitude characteristic and a linear delay characteristic. The 
final distortion after equalization with an JV-tap equalizer is shown in 
Fig. 8 for two different values of peak delay. With a peak delay of 5 
pulse intervals the initial distortion D = 1.4. In this case it happens 
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that the equalizer converges and is optimum. The more taps A r which 
are used on the equalizer, the lower the final distortion. 

When the peak delay in this example is increased to pulse intervals, 
we encounter a radically different behavior. The equalizer still con- 
verges to the tap settings which satisfy h n = for | n | ^ N/2, n ^ 0, 
but this solution no longer minimizes distortion. The initial distortion 
here is about 2.1 and after equalization with a 6-tap equalizer the final 
distortion is over 3. Clearly this is not optimum, since the zero tap set- 
tings gave better performance. As is shown, increasing the number of 
taps results in larger residual distortion. 

The equalizer's failure in this latter case is more clearly illustrated in 
Fig. 9. The impulse response before equalization looks harmless enough, 
but after the equalizer has forced 3 zeros on either side of the response 
peak (b'-tap equalizer) a large side lobe is created outside the equalizer 
time range on the right-hand side of the response. This side lobe con- 
tains more distortion than was present in the original response. In this 
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Fig. 8 — Example of equalizer failure — linear delay. 
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Fig. 9 — (a) Impulse response fur channel with linear delay, m = 6; (b) after 
equalization with 6-tap equalizer. 

example minimum distortion settings of the equalizer do not set zeros 
at the samples h n for | n | ^j (N/2), n ^ 0. The actual optimum settings 
may be obtained by methods we will discuss presently, and for com- 
parison purposes the minimum obtainable distortion for this example 
is plotted in Fig. 8. 



3.4 Strategy When D > 1 

We described the situation when the initial distortion was less than 
100 per cent as "loosely coupled." In this range many iterative schemes 
can be devised which converge to the optimum. Conversely, when 
A, > 1 the tap gains become more strongly interdependent. Also, the 
optimum may shift so that it is impossible to simply instruct the auto- 
matic equalizer to force A r zeros in the impulse response at the loca- 
tions of the N variable taps of the equalizer. Thus the equalization 
strategy for the general case becomes more subtle and complicated 
than the simple strategy previously explained. 

Fortunately, Theorem II describes D (using center-tap normaliza- 
tion) as a convex function of the tap gains. Therefore the distortion has 
a single minimum, and mathematical programming methods may be 
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used to locate the optimum tap gains. A steepest-descent method may 
be applied to D [see (14)]. The gradient of D is 

VD = E ^ aj (33) 

j=-N/2 OCj 

d ^ = E' (*-/ - *«*-,) sgn h n . (34) 

dC; « — » 

If each tap gain cj is incremented proportional to its gradient component 
(34), the system will eventually approach the optimum tap gains. The 
dotted lines showing minimum distortion in Fig. 8 were calculated using 
this strategy on a digital computer. * Such a strategy can be implemented 
with a modification of the circuitry previously described. In this im- 
plementation the term involving .r n .r_, in (34) is neglected as small in 
comparison to .»„_> . The remainder of (34) may be derived physically 
by a two-pulse cycle. The first test pulse is sliced to obtain the sequence 
{sgn h„] which is stored in the shift register. This sequence is then 
used to control the polarity of the gains at each tap. A second test 
pulse is transmitted and the transversal filter serves to multiply the 
input sequence [x n ] by the tap gain sequence {sgn h n \. The output volt- 
age is approximately the sequence {dD/dcj\ from (34) and may be 
used to digitally increment the tap attenuators. 

In any such scheme it is now necessary to have a transversal filter 
about twice as long as the number of variable taps to be used. This is 
because the sum in (34) is infinite, but practically speaking an A/-tap 
transversal filter will affect the impulse response for not more than 
2NT seconds. Thus the test pulses must go through a 2JV-tap delay 
line, all of whose taps are equipped to handle the ±1 gains which store 
the sequence {sgn h n }. However, only the inner N taps need have 
associated variable attenuators. This complication comes about because 
the distortion depends on what happens outside the NT range (witness 
Fig. 9!) as well as what happens insidc.This information must be meas- 
ured by "listening posts" established by taps on the delay line outside 
the normal range and then taken into consideration in incrementing the 
variable tap gains. 

IV. THEORETICAL PERFORMANCE OF THE EQUALIZER 

4.1 Design and Performance Parameters 

Our attention will now be confined to the equalizer described in 
Section 3.2 and illustrated in Fig. 4. In this fairly simple system there 

* The distortion minimization can also be formulated as a linear programming 
problem, so that using a digital computer the exact optimum can be reached in a 
finite number of steps. 
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are only two parameters which the designer has under his control to 
affect the performance and cost of the system. These are: 

N — the number of variable taps on the transversal filter. 

A — the step spacing on each electronically controlled attenuator. 
(A lore generally this is A,- , since different tap positions may take 
different-sized steps. ) 

The economics of the choice of these two parameters may be readily 
appreciated. The cost of the equalizer is nearly directly proportional to 
N, since not only docs a larger N entail a proportionally longer delay 
line, but also proportionally more logic circuitry. A major portion of 
this logic circuitry is taken by the reversible counters for setting the 
attenuator tap coefficients. To decrease the step spacing A, each of the 
counters must be augmented by additional stages. 

In judging the performance of the equalizer we shall be interested in 
the following two parameters; 

D r — the final distortion or residual distortion after equalization. 
T, — the settling time or the time required to set the equalizer during 
the training period. 

Obviously, the smaller the residual distortion D r the better the data 
system will perform, but on the other hand it is too expensive to at- 
tempt to reduce this residual distortion much below the required toler- 
ance for a given system. The settling time T, on an experimental im- 
plementation of the automatic equalizer has been on the order of a 
second. When compared with the time required for establishing the 
call and acquiring timing synchronization this seems negligible. How- 
ever, we shall find that when greater accuracy is required the time T, 
can become quite appreciable. 

The performance parameters D r and T, depend on the design parame- 
ters N and A and upon the channel to be equalized. Since it has been 
common to describe the channel by frequency-domain characteristics 
we shall do so here. The channel characteristics of interest are: 

A(w) — the amplitude characteristic of the system, including trans- 
mitter shaping filter and the attenuation characteristic of the channel. 

/3 (w) — the delay characteristic of the channel. 

S/N — the signal-to-noise ratio of the channel. The noise is assumed 
to be Gaussian and white since the effects of impulse noise are not 
particularly important in the automatic equalization system. 

Fig. 10 shows how the parameters N and A affect the performance of a 
noiseless system example. In these curves the distortion is plotted as a 
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Fig. 10 — Distortion vs time. 

function of time. It can be seen that increasing the number of taps N 
results in a lower residual distortion without affecting the settling time 
of the system. After the required settling time has elapsed, the system 
reaches a limit cycle where all taps oscillate one step up and down 
about the optimum values. Decreasing the step spacing A decreases 
the oscillation of distortion and increases the settling time required. 
When" noise is introduced into the model, the curve of distortion versus 
time becomes a random walk with final values of distortion becoming 
random variables. 

The residual distortion is composed basically of two components. 
One component is the theoretical minimum distortion for the given chan- 
nel and length N of transversal filter available. This corresponds to the 
case where S/N — > °° and A — » 0. No settings of the filter can reduce 
the residual distortion any further. This distortion component will be 
designated D c . It is a function only of the channel characteristics 
A (w) and /3' (co) and of the number of taps N. 

The second component of residual distortion arises from our inability 
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to reach optimum tap settings because of the oscillation of the final 
tap settings due to noise and finite step spacing A. This component is 
approximately independent from tap to tap. On a per-tap basis it is a 
function only of step spacing A ; and signal-to-noise ratio. This com- 
ponent will bo designated D g , where D s is the contribution from a single 
tap. Thus for an A^-tap equalizer the distortion may be resolved as 
follows. 

D r = D e (N,A,P') + Z' D.(A,,S/N). (35) 

i— N/2 

The subscripts "c" and 'V stand for channel distortion and system 
distortion respectively. 

In the next section we will discuss generally D c , the channel distor- 
tion, and show curves relating D c and N for various shapes of amplitude 
and delay distortion. The subsequent section then deals with the system 
distortion D a . Curves are shown relating D s and A for various signal- 
to-noise ratios. The question of settling time is also discussed in this 
latter section. 

4.2 Residual Channel Distortion 

When Do < I, the minimum distortion is obtained by setting h„ = 
for | n | ^ N/2, n 7^ 0. The remaining distortion we describe as D c , 
the residual channel distortion. 

D e = .- E I hn |. (36) 

lUt In I >JV/2 

This expression for the residual channel distortion is easily calculated 
on a digital computer from .V, A (co), and /3 (u) by first computing the 
response samples j.r„} from the Fourier transform of the channel's 
frequency characteristic and then solving the TV simultaneous equations 
/?„ = for | n | ^ N/2, n ^ 0. A number of curves obtained by this 
procedure are presented in Figs. 11 through 13. In all of these figures 
the transmitter shaping characteristic was raised cosine with a 50 per 
cent roll-off. (Full raised cosine shaping gives very similar results.) 
The reference time was taken at the peak of the response x(t). This is 
not necessarily optimum, but usually is fairly close to optimum. 

Fig. 11 shows the residual channel distortion for parabolic delay over 
a wide range of equalizer size N. Fig. 12 is a similar presentation of the 
residual distortion resulting from linear delay. In Fig. 13 the channel 
has no delay distortion, but has an attenuation characteristic which 
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Fig. II — Residual channel distortion for an iV-tnp equalizer with parabolic 
delay. 

falls off with various slopes. These curves are intended to answer the 
question of how well a given transversal filter can equalize a given 
type of distortion. These curves represent theoretical minima. To see 
how well an actual equalizer can approach these values, the system 
distortion D, for the given A and S/N must be added. From (36) the 
residual distortion consists of all the response samples h n removed from 
the center of the response by more than nT/2 seconds. This is approxi- 
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mately composed of two components: what distortion was already out 
there in the original channel response x(t), and what distortion has been 
squeezed out into this range by the equalization process. We know from 
echo theory that the former component results from delay and amplitude 
ripples of greater than N/2 cycles in the bandwidth and increases ap- 
proximately linearly with the amplitude of the ripple content. The latter 
component of the residual channel distortion which is squeezed outside 
the transversal filter range is more difficult to conceptually visualize. 
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Fig. 12 — Residual channel distortion for an JV-tap equalizer with linear delay. 
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Fig. 13 — Residual channel distortion for an AMap equalizer with attenuation 
distortion present in the transmission channel. 

We can show that this component of D c increases approximately with 
the square of the delay or amplitude variation in the channel. 

Removing the large terms involving c and x , which are unity, from 
under the summation in (9) we arrive at 



N/2 

K = X„ + C„ + Zl CfCn-} 

j JV/2 



(37) 
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Iii order to make li„ = for — N/2 S n ^ N/2, n 5^ 0, it can be seen 
that a first approximation for the tap setting cj is 

cj « -xj . (38) 

After equalization we will have h„ = inside the range of the filter. 
Outside the range we will have approximately, using (37) and (38) 

(39) 





\ 2 


h„ » .. 


■„ - X) •'•>•'■»-. 


|»|>#/S 


; — A72 




i^o.n 



The first term in (39) is the initial distortion term and the second is 
distortion outside the range of the filter generated by the equalization 
process. Since the samples .v„ vary approximately linearly with ampli- 
tude and delay ripple amplitude, it can be seen that the two components 
of h„ vary linearly and quadratically with ripple amplitude respectively. 
The curves in Figs. 11 through 13 show that the linear term domi- 
nates in the case of parabolic or linear delay in the channel. This indi- 
cates that very little additional distortion in the range | / | > NT/2 is 
being pushed out by the equalization process. In Fig. 11 directly be- 
neath the N = 2 curve is plotted the initial distortion outside the range 
of this particular equalizer. This dotted curve marked "approximate" 
indicates the quantity 

D r (approx.) = £ I** |. (40) 

I n I > 1 

Iii the upper range of the N = 2 curve the second component involving 
"squeezed-out" distortion becomes important. 

As long as the linear term dominates we can think of the transversal 
filter as completely equalizing any delay or amplitude variation of less 
than N/2 cycles in frequency content. For example, a Fourier expansion 
of the parabolic delay reveals the major content at low frequencies; 
thus few taps are required to equalize parabolic delay. The higher- 
frequency content of the parabola is unequalizable by a short trans- 
versal filter. 

Frequently the delay and amplitude characteristic of a channel con- 
sist of approximately sinusoidal ripples. This is generally the case 
when an equalization network has been incorporated in the channel. 
A good rule of thumb is that it takes twice as many taps on the equalizer 
as there are ripples in the bandwidth. Thus the better a channel has 
been previously equalized the longer a transversal filter will be required 
to further improve equalization. Frequency- and time-domain equaliza- 
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tion are very similar in the respect that they both leave ripples of delay 
and amplitude of higher frequency after equalization. 

4.3 Residual System Distortion 

The term D„ , residual system distortion, results from the inability 
of the equalizer to reach the actual optimum tap settings. In the case 
of very high signal-to-noise ratio the value of D R is obvious. The tap 
cj will end in a limit cycle of one step Aj about the optimum tap setting. 
In one position 

Cj = c> (optimum) + c (41) 

while in the other position 

Cj = c } r (optimum) + e — A, . (42) 

Assuming that A, > | e | and neglecting the last term in (37), we ob- 
tain the average system distortion for this case. 

A(A;, (S/N) = ») = \Aj. (43) 

Thus the system distortion is one-half step for high S/N. However, a 
system will very seldom operate in a true high-S/N environment be- 
cause the tap steps themselves are usually well within the noise. For 
example, with a A of 0.01 the step-to-noise ratio is 40 db below the 
signal-to-noise ratio. With noise the taps end in a random walk instead 
of the limit cycle of the noiseless case, and the average system distor- 
tion will be considerably higher than that given by (43 ) . 

Let's examine the behavior of an individual sample hj when noise is 
present in the channel. The receiver then bases its decision on whether 
to advance or retard tap c 3 by the amount A on the sign of the quantity 
(hj + rij), where n, is Gaussian with mean zero and variance a 2 , and is in- 
dependent over index j and from test pulse to test pulse. Assuming that 
the sample hj is affected only by tap setting Cj [equivalent to drop- 
ping the last term in (37)] we write 

hj&Xi + c,. (44) 

Since c ; takes on only values which are integral multiples of A, we can 
without any particular loss of generality quantize hj in steps of A. The 
behavior of each of the samples hj is similar, so in order to keep con- 
fusion to a minimum we shall drop the "j" index and define 

Prob [h = kA, after m test pulses] = p(k,m). (4f>) 

Now we can write a difference equation for this probability as follows 
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p(k,m + 1) = p(k - l,m)P(k - 1) + p(k + l,m)Q(k + 1) (46) 

where P{k) and Q(&) are the probabilities of noise being less than or 
greater than — A-A respectively. 

P(k) = [ -7—- exp - (x 2 /2a 2 ) dx = Hi " Erf (JbA/V§er)] (47) 

Q{k) = 1 - P(fc). (48) 

Initially the value hj starts at position Xjtt I A so that we can use the 
initial condition 

Pirn = {J * * ; ; (49) 

The difference equation (46) then defines a probability distribution 
which spreads out as time (m) progresses and eventually ends in a 
stable symmetrical distribution centered at zero. [Actually there are 
two final distributions reached: one for m even and one for m odd. Since 
the system could be turned off in either state with equal likelihood, 
these distributions must be averaged. This effect is equivalent to the 
averaging of (41) and (42) in the noiseless case.] 

A number of these final distributions were computed using (4(i), 
(47) and (48) on a digital computer. The contribution to the residual 
distortion owing to oscillation of tap Cy is the random variable \hj\. 
The system distortion Z),(A> , S/N) is defined as the expected value of 
\hj\ 

D. = E[\hi\] = A y £|A-|p(A-,oo). (50) 

k 

A number of curves relating D s , A and signal-to-noise ratio are shown 
in Fig. 14. Observe that these curves are nearly piecewise linear with a 
break point approximately where A = a. Above this point the high 
signal-to-noise condition prevails and D„ tt \A. Below this point the 
steps A are within the noise and the slope of the curve changes. By 
making a low signal-to-noise approximation we can derive equations 
for these curves. 

For A/<r small we use the first term of the series 

Erf(x) = ^.[i-£ ; + £-Jl+...l (5i: 



and obtain 
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Fig. 14— D, vs A for various S/N ratios. 

Using this transition probability in the difference equation (46) we 
obtain an equation similar to the classical problem of the random walk 
of an elastically bound particle: 



p(k,m + 1) = p(k - \,m) - 



n _ (k - 1)A H 
[_2 \/2^r <r J 



n (k 4- 1)A" 



(53) 



An exact solution to this difference equation assuming the initial condi- 
tion (49) is given by M. Kac in Ref. 3. Since this solution is more 
complicated than the asymptotic distribution obtained as A — > and 
is no more useful for our purposes, we will not repeat Kac's formula here. 
It is simple to derive expressions for the mean and variance of hj 
after m test pulses have been transmitted. Using (53) we obtain for 
the mean value 



K[h (m)] = A J2kp(k,m) 

A 



(54) 



E[h(m)] = A - + 



G 



+ 



A 2 



\/2tt 



-\/2ir <r_ k 

■k[p(k - \,m -1)4- p(k + \,m - 1)] (55) 

- Z k 2 [p(k + l,m - 1 ) - p(k - \,m - 1)]. 

<r k 



AUTOMATIC EQUALIZATION 575 

The various summations in (55) can be rearranged with changes in 
variable to give 



(58) 



E[h(m)] = E[h(m - 1 )] f 1 - ^=-) ■ (56) 

Since the initial value of hj is Xj ~ lAj , the average value of hj after 
m test pulses is 

/ 2A,- \'" 

'* , " ,)| = M '( 1 "V^)' (57) 

One can derive a similar expression for the variance of hj after m 
test pulses 

E[hj 2 (m)} = (lAj) 2 (l ~-^lf 

As m — + oo the final distribution is obtained and 

HmiflMro)] = (59) 

m-*oo 

lim E\hi\m)\ = ^Ajcr. (60) 

Equation (00) is particularly interesting, since the variance of hj is 
proportional to the square root of the noise variance a 2 . 

Now suppose the tap-to-noise ratio A/a tends to zero and test pulses 
are sent at a rate 1/r which tends to infinity. Using the difference 
equation (53) we can derive a differential equation for the probability 
density of the sample hj after / seconds have elapsed: 



dt V 2tt <tt L dh J 2t dhr 



A solution to this equation, attributed to Lord Rayleigh, is given in 
Ref. 3. Assuming hj starts at Xj , the probability density after t seconds is 

P(hj , t) = /T .) , T< exp [-(hj - Xje-"Y/2a\l - •"*»')], (62) 

where a is the same as E[hj (<» )) in (60) 

lv Aj* (63) 



4 
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and 

2A,- 



y = "THr 



V27T 



(64) 



As t — > oo the density (62) tends to Gaussian of mean zero and 
variance a. We can thus easily derive the low signal-to-noise ratio value 



of/).. 



D, = E[\ hi I] = 2 I —d=~ exp - {hf/2o?) dh, (65) 

•mi "V ^tt a 

D. - \/2A« = (o-Ay/V^)' (66) 

D. « 0.633 s/TKj . (67) 

The straight-line portions of the curves D, versus A in Fig. 14 match 
nearly exactly with (67). 
In summary, we have found 

f Ay/2 if Ay > a 

Z>.« ^_ (68) 

l0.633-\AAy if Ay < a. 

The important fact about (68) is that the residual system error goes 
down as the square root of both tap spacing A and the standard devia- 
tion of the noise a. Thus once the step spacing and noise are comparable, 
cutting the steps A by a factor of ten cuts the residual system error due 
to tap uncertainty by about 3.16. An interesting sidelight is that cutting 
A by ten might seem equivalent to averaging ten samples of hj before 
taking one step of A. In the latter case the noise a would be cut by 
\/l0 = 3.16 and the residual system distortion D„ cut by only V3.16 = 
1.77. However, as might be guessed this latter technique will require 
less settling time than the former technique using 0.1 A and moving 
ten times as fast. 

This leads us into an abbreviated discussion of settling time. The 
time required to settle is usually determined by the largest distorting 
sample rcy for j ^ 0. If the step-to-noise ratio A/a is large, then the 
settling time is obviously 

T. = x jmax /A;. (69) 

When the noise becomes important, then we need to make some sort of 
arbitrary definition of settling time. This definition could be based on 
the time required for the mean of hj or the variance of hj to reach some 
predetermined position or percentage, or it could be based on the time 
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constant I/7 in the density given by (62). All these types of definition 
lead to similar expressions differing chiefly by constants, so we'll use 
the simplest and define 

T. = I/7 = y/fr<rr/2A t . (70) 

Recall that r is the interval between test pulses. This expression is 
independent of Xj mKX for low tap-to-noise ratios. Using a binomial ex- 
pansion of the term to the with power in (57), (70) can be derived as 
the point where the first two terms of the expansion for E[hj(m)] cancel, 
doing back to our previous example, where we compared decreasing 
the step spacing A with averaging several samples of hj , we can now 
show that cutting the tap spacing A by a factor of N is exactly equiva- 
lent in settling time and system error to averaging N 2 consecutive sam- 
ples of hj . This is evident from the proportionalities involved in (69) 
and (70). In either case the same amount of time is required to achieve 
a given level of accuracy in equalization. 

V. FREQUEXCY-DOMAI.V CONSIDERATIONS 

5.1 Frequency-Domain Relationship 

The equalizer that has been described is intended for the correction 
of distortion in digital data transmission. This equalizer is strictly a 
time-domain device which corrects the pulse response of the channel 
to the best ability of a finite-length tapped delay line. The question 
which most frequently arises concerning its operation asks what happens 
to the frequency characteristics of the channel as a result of the time- 
domain equalization. This question is asked not only out of curiosity 
and because engineers tend to think in terms of the frequency domain, 
but also because the time-domain equalizer is sometimes considered 
for the equalization of analog channels. 

In this section we will develop a general formula for the frequency- 
domain characteristics of an equalized channel in terms of the un- 
equalized characteristics. This formula allows us to compute final 
characteristics and to make a few general observations about the re- 
lationship between initial and final characteristics. In addition to 
these results we are able to derive conditions as to when a channel 
may be equalized successfully, and we find one additional technique 
of computing optimum tap settings for long equalizers. 

Henceforth we will assume an infinite-length transversal filter. If 
the equalizer is long enough to do its job properly, the final characteristics 
will closely approximate the infinite length characteristics we will 
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derive here. An outline of this derivation is as follows. If the equalized 
pulse response is to have zero distortion, it must have Nyquist frequency 
characteristics implying symmetry in real and imaginary components. 
Knowing the channel characteristics and the type of characteristic 
capable of being assumed by the equalizer, we show there is only one 
such Nyquist characteristic the product can assume. This characteristic 
must be the final frequency response of the equalized channel. 

The impulse response of the channel in terms of the amplitude, 
A(io), and phase, /3(o>), is 



hi 



t) = - [ A(u) cosM - 0(«)]dw. (71) 

We assume that the time base has been adjusted by the removal of a 
flat delay (linear phase) component from /3(w) so that the peak of the 
output response occurs at time zero. For zero distortion we require 

h n = = - / A(u) cos [nuT - p(ta)]du all n, n ^ 0. (72) 

Changing (72) to use real and imaginary components we arrive at the 
equivalent condition 

[ A x (u) cos ntoTdo) = all n, n 9± (73) 

•'o 

\ AM sin nojfrfcj = all n (74) 

where 

A m (a) = A(«)cos0(a>) (75) 

A v (<a) = .4(co)sin/3(u>). (76) 

Let us assume that A («) = for u > 2t/T, i.e., the channel has no 
frequency component higher than twice the Nyquist band for its signal- 
ing rate. It would be an unusual case if this were not true. We now make 
a change in variables to shift the origin to the Nyquist frequency ir/T 
and redefine the characteristics about this frequency. 

A x («) = A.L> + £\ (77) 

AM = A f (« + j). (78) 



./: 
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(111 general, a circumflex on any variable indicates that it is defined 
about the frequency r/T as origin.) With this change the conditions 
(73) and (74) become 

■«7 T 

A x {ta) cos maTdu = all n, n 5* (79) 

— r/r 

/T/T 
A y (u) sin ntaTdu = all n. (80) 

■T/T 

We also have the normalizing condition for the center sample. 

fo = 1 = _ / A s (u)du. (81) 

IT *" — x/ T 

E(iuation (79) says that A x {u) must be orthogonal to all cos ruaT 
between —ir/T and t/T. This implies that its Fourier expansion con- 
sists of a constant to satisfy (81 ) plus any arbitrary sine components. 
In other words it must be a constant plus any arbitrary odd function. 
Similarly, A v (u) must be any arbitrary even function to satisfy condi- 
tion (80). Simply stated, the conditions are then 

A s (u) = (772) + odd function (82) 

A v (u) = even function. (83) 

These conditions were first derived by Nyquist, 4 although engineers 
are generally more familiar with the case of flat delay where (82) and 
(83) reduce to the statement that .4 (u) has Nyquist symmetry. 

Now that we see the conditions that the product of the equalizer 
and channel responses must meet for perfect equalization, let's look 
at the type of response the equalizer alone can have. The Fourier trans- 
form of the equalizer's time response is 

C( w ) = E m c„e- inuT (84) 

where the c,/s are the tap gain settings. As before we shift the origin 
of definition for C'(w) to t/T. The change in variables and definition 
gives 

<?(«) = (?,(«) +><?,(«) (85) 

with 

Cx(o>) = Co + £ (-l)"(c„ + c_ n ) cos nwT (80) 
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C,M = £ (-l)"(c« - c-n) sin ncoT. (87) 

n=l 

Observe that C x (u) is an even function and (?„(«) is an odd function. 

Now let us suppose that the unequalized channel frequency response 
in terms of real and imaginary components defined about w = t/T is 

I(co) = £.(«) + A(to). (88) 

The product of the unequalized frequency response A(u>) from (88) 
and the equalizer response C(a>) from (85) must satisfy the odd and 
even requirements on its real and imaginary components as given in 
(82) and (83). After separating the product A(w)C(u) into real, 
imaginary, odd, and even components, it is possible to arrive at simul- 
taneous equations for C x (u) and C„(a>). The details of this process are 
quite straightforward and have been relegated to Appendix B. 

The final equalizer response is found in terms of the even (labelled 
with an "e" subscript) and odd (labelled with an "o" subscript) com- 
ponents of the unequalized channel response. The equations derived 
are 



<?.(«) = -r 



2 AmM (89) 



AJ(<*) + AJM 

C y (<a) = 



T X ( \ 
- g A v°^> ((jo) 



A xe -(w) + A "(«) ' 



where 



i XP (o,) = Hi.(«) + ix(-a.)i (si) 

AM = *[£•(«) - !«(-«)]. (92) 

These equations determine the real and imaginary components of the 
equalizer frequency response. 

Using this response for the equalizer, the equalized channel response 
becomes 

2 /V \tt \ |"l I A xe (u))A xo (a)) + Ayo(ia)Aye(h))~\ 

- C{a>)A(u) = 1 + 1 2 

* L ^ixe (w) + A yo (U) J 

V J«J / 

. r A I(; (fa))^ ye (^) — A„ (co)A IO (co) "| 

J l AJ(u>) + A„ 2 (u) J 



which is the result we have been seeking. 
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5.2 Interpretation of the Frequency-Domain Relationship 

The frequency response of the equalizer [from (89) and (90)] is 

272 

C(u) = * 2/ x , X 2/ x Uxe(w) - Jii tf0 (w)]. (94) 

Recall that A xc (u) is the etraw component of the real part of the channel 
response taken about the Nyquist frequency, r/T. Similarly A v „(co) 
is the odd component of the imaginary part of the response. Notice 
that C(co) does not depend on A xo (u)) or A y( .(cc), so that these com- 
ponents may be specified arbitrarily without changing the equalizer 
settings. This points up the difference between frequency-domain 
equalization and time-domain equalization. Our data distortion criterion 
is such that we don't care about the components A x0 (w) and A ve (co) 
of the frequency response. However, these components affect the shape 
of the final equalized characteristic, as can be seen from (93). 

It is clear from (94) that the channel camiot be equalized if A xe (co) 
and A yo (o)) are both zero for some to, ^ co ^ t/7 1 , in which case C(co) 
is unbounded. If this does not happen, practical considerations indicate 
that the channel is capable of being equalized. (These practical con- 
siderations require that both [A(u)C(c*)] and its Fourier transform are 
continuous and absolutely integrable.) 

Equalization Condition: 

A channel with complex gain A (to) is capable of being perfectly 
equalized with an infinite transversal filter if and only if the even com- 
ponent of the real part of A (co) and the odd component of the imaginary 
part of A (co) do not simultaneously vanish for some co, ^ co ^ t/T. 

Note that a channel can be equalized even if it transmits no energy 
in some interval [coi , wj, a> 2 < tt/T, so long as the missing energy com- 
ponents are replaced in a symmetrically located region above ir/T, 
i.e., [(2tt/T) - w 2 , (2ir/T) - co,]. 

Now let's go back to (93) to try to get some feel for the shape of the 
final response. The equation itself is not complicated, but it unfortunately 
requires us to break A (to) into even and odd components about the 
Nyquist frequency v/T and then further to break these components 
into real and imaginary parts. By this time we have almost no idea what 
the final response will be when we put everything back together as in 
(93). To take a special case which simplifies this process, suppose that 
the channel has perfectly flat delay, so all the "y" components are zero. 
Then 
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A(»)CM - IT 1 + 4=7^1 ■ (95) 

2 L A xe (a))J 

The first term represents a rectangular characteristic and the second 
term is an odd function about the frequency t/T. Thus we have a 
familiar Nyquist characteristic which is easily determined from the 
initial characteristic A(u). Fig. 15 shows a sample case of this type 
having flat delay (linear phase). The components l M (w) and l IO (co) 
and the equalized spectrum arc shown for this example characteristic. 
With a little study it is possible to get a good feel for the sort of final 
characteristic which is obtained by equalization in this purely real case. 
However, the introduction of nonlinear phase results back in the compli- 
cated expression (93). 

Typically, the channel to be equalized cuts off somewhere before 
oj = 2t/T, so there are no energy components in the interval 
[(t/T) + u*c , (2r/T)}. In this case we have 



l*e(«) = 
A ye (w) = 



—A xo (a) 

— Ay (u) 



l le (co) = 1,0 (w) 
lyefa) = lyo(w) 



tt/T ^ CO ^ 



-tt/T ^ CO ^ -co c 



Substitution of (96) and (97) into (93) shows that 

!(«)<?(«) = T/2 for -ir/T ^ co ^ -« 



(96) 



(97) 



(98) 




Fig. 15 — Equalization of amplitude distortion. 
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Which moans 

A(co)C(co) = I for ^ co ^ ~ - Ue . (99) 

We can boil this equation down into the most significant observation 
we can make about the frequency-domain relationship: 

If the channel response cuts off at the frequency (t/T) + co c , then 
the channel must be equalized to constant amplitude and linear phase 
in the region ^ co ^ (k/T) — co c . 

This observation is important in the equalization of analog channels 
to flat amplitude and linear phase. In order to get the largest possible 
interval of perfect analog equalization we should arrange the tap spacing 
T so as to approach the Nyquist interval corresponding to the channel 
cutoff. However, if the channel cuts off before t/T then by the previous 
results the channel is unequalizable. The closer T is to a Nyquist interval 
the more taps will generally be required to effect a good equalization. 
One final note to add to the confusion- — if co c is large very little of the 
spectrum will be equalized flat. If the tap spacing 7' cannot be changed 
we can increase this region of flat equalization by inserting a low-pass 
filter during the equalization period which has a cutoff frequency close 
above t/T. The filter may then be removed after the equalizer tap gains 
have been set. 

Some of the principles involved are exemplified in Fig. Hi, which 
shows the equalization of a channel which has a linear delay charac- 
teristic (quadratic phase). Initially the amplitude response of this 
channel is a 50 per cent raised cosine response cutting off at co = 3t/2T. 
This amplitude characteristic is a Nyquist shape in the absence of delay 
distortion. After equalization both amplitude and phase are flat from 
w = to u = 71-/27', as required by our previous observation. From 
a) = t/2T to co = 37r/27' the phase still appears parabolic. In this region 
there has been an interaction between phase and amplitude which has 
resulted in a change in the amplitude characteristic away from its 
Nyquist shape. The combination of equalized phase and amplitude in 
this region is such as to satisfy conditions (82) and (83), although this 
is not evident from casual inspection. 

One additional important usage of (93) for C(co) is in the calculation 
of optimum tap settings for the equalizer. Starting from the unequalized 
channel pulse time response samples \x n ) we construct ^4 xe (co) and 
A„o(co) exactly as in (8(5) and (87) for C x (w) and C y (co), i.e., let 
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AMPLITUDE**' \ 


JPy. \ 



1/2T 
FREQUENCY 



i/T 



Fig. 16 — Equalization of linear delay distortion. 



A xv (w) = .r„ + 2 (-!)"(*„ + ■'•-,,) COS nuT 



(100) 



it.(«) - E (-!)"(•'•« - x- n ) sin n«5T. 



(101) 



The other components, A xo and A yc , are of no concern to us, since they 
affect neither \x„\ nor the equalizer settings. Now calculate C(u) from 
(93) and evaluate its Fourier transform at time nT. Each of these 
values may be identified with tap settings through (9(>) and (97). This 
procedure may seem complex, but it is by far the fastest way to calcu- 
late tap settings for long transversal filters on a digital computer. Time- 
domain techniques involve the solution of simultaneous equations or 
iterative methods depending on the input pulse. The time required for 
this type of minimization usually increases with the square of the num- 
ber of taps involved. As each new tap is added, all the previous tap 
settings change to readjust for the new minimum for the increased filter 
length. 

On the other hand, using the frequency-domain technique the time 
required for computing tap settings increases linearly with the number 
of taps involved and each new tap setting does not change any of the 
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previous settings. The catch is that the settings calculated by the 
frequency-domain method apply only to an infinite-length line. For 
lines of finite length the settings calculated by frequency- and time- 
domain methods will differ. The time-domain settings are calculated 
on the basis of minimum distortion — a quantity defined on the basis 
of time samples. The frequency-domain settings for a finite line are a 
root-mean-square approximation to the final equalizer frequency char- 
acteristic for an infinite line. As the number of taps goes to infinity the 
tap settings computed by each method approach the same value. For 
any finite line the time-domain settings are better (lead to less distor- 
tion), since they are by definition optimum. However, for equalizers of 
about 12 taps or more the results are practically the same. (Indeed, one 
should remember that the distortion criterion itself is somewhat arbi- 
trary.) Thus the frequency-domain technique described here is a fast 
and accurate way of computing tap settings for long equalizers. Un- 
fortunately the method does not lend itself to implementation, since it 
involves a number of difficult operations, i.e., division, Fourier inversion. 
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APPENDIX A 

Statement and Proof of Theorem I 

Theorem I 

If D a < 1, then the minimum distortion D must occur for those N tap 
gains which simultaneously cause h„ = for all n C K N ,n^0. 

Proof 

We prove this theorem by assuming a minimum has occurred at some 
point other than that specified in the theorem and showing this assump- 
tion leads to a contradiction. Thus we assume that D is at a minimum 
for some sequence of tap gains \cj\ and that h k 9* for some A- c K N , 
k 5^ 0. Now we will show that there exists another sequence \c*\ for 
which D* < I) and hence the contradiction. 



*=*» (l04) 
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Let 

Cj* = Cj j C K N , j ?± 0, j ^ k 

(102) 
c k = c k - A sgn h k 

where 

|| h k | > A > (103) 

which is possible, since by hypothesis ft* ^ 0. The value of distortion 
corresponding to the tap gains {c,*J is, from (14) 

d*= E' I £' Cj (.,-„_, -.»•„.,•_, ) + .,-„ 

n— oo ;c 
J** 

+ (c* — A sgn />A-)(.r„_A- - .r„.r_ A .) + ar* |. 
Using (13) this reduces to 

D* = Z' | h„ - A sgn k k (x n -,> - .r„.r_ A .) | (105) 

00 

O* = 53' I /,„ - A sgn ft*(x„_* - .r„.r_ A - ) | 

:sr (ion) 

+ | h k — A sgn h k (l — XtX-k) |. 

We concentrate for the moment on the second term of (10G) and use 
h h = \hk\ sgn fh- to obtain 

| h k - A sgn Ml - xtx-t) | = | | h | - A(l - ;r,.c_,) |. (107) 

But | XkX-k | < 1 since 

Do = £' \x n \ < 1. (108) 

II =— 00 

Therefore 2 > (1 - .r ft .r_*) > and, by (103), 

| A* | - A(l - awr-t) > 0. (109) 

Thus we are able to drop the absolute value brackets around the second 
term in (106) and write 

D* = fy | h n - A sgn h k (xn-k - .r„.r_*) I 

*r (no) 

+ \h k \ - A(l - .iv-*) 
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D* £ E' | *. I + A jy I ,- n _ t - x n x. k 



II =—00 

II ^/.- 



- A(l - .r,.r_,) 



I =—00 

n?4b 



(HI) 



(112: 



(113) 



- (1 -.r,.r_,)l 

D* ^ D + A {A, - | .r_, | + | x_ k | (A - | x k \ ) 

-1 + | x k I I .r_A- | j 

Z>* g D - A(l - D )(l + I.T-.I ) (114) 

and since jDo < 1 

D* < D (115) 

which is a contradiction, since we assumed that D was originally at a 
minimum. This completes the proof of Theorem I. 

APPENDIX B 

Derivation of Equalizer Frequency Response in Terms of Unequalized 
Channel Frequency Response 

The equalizer frequency response taken about the Nyquist fre- 
quency w/T is 

c(w) = €„(<*) +#,.(«) (ii6) 

where we use the subscripts "x" and "y" to denote real and imaginary 
components and the subscripts "e" and "o" to denote even and odd 
components. Breaking the unequalized channel frequency response 
similarly into its components gives 

l(co) - A xv (u>) + A xo (u>) + jA m (o,) + jA yo (u>) (117) 

with 

AM = ![£.(«) +£.(-«)] (H8) 

4,o(«) = }[!.(«) -ix(-co)] (119) 

etc. 

We are now ready to put together the equalizer response and the 
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channel response to get the equalized channel response. Using (117) and 
(116) for this purpose we obtain 

+ iC x „(w )](?„„ (to) + [i ve (w) + i„o(w)]C IC (a>)}. 

This equation must meet the conditions (82) and (83). If 8„ is an arbi- 
trary odd function, condition (82) dictates that 

Re [A («)£(«)] = (T/2) + 5 (121) 

[ixo(co) + i«(«)]&.(«) - &■(«) + A yo (co)]C yo (u>) 

(122) 
= (T/2) + 5 

!•(•)&.(«) - l»o(«)(?»a(«) = r/2 (123) 

£«,(«)&.(«) - ^„ (a J )C„ (a,) = 5„ . (124) 

Since 8 is an arbitrary function, (124) is automatically satisfied. 
Let 8 C be an arbitrary even function. Condition (83) now requires 

Im [!(•)<?(«)] = 5 e (125) 

[*.(•) + ixo (»)#..(•) + &.(«) + i tf o («)](?«, (•) - «« . (126) 
The even and odd equalities from (126) are respectively 

A X(1 (o:)C yo (co) + !„(«)<?«(«) = 5 U (127) 

I«(«)tf«o(«) + l„o&e(«) = 0. (128) 

Again (127) is satisfied trivially and we are left with two equations, 
(128) and (123), in the two unknowns <?„(«) and C„ (w). These equa- 
tions may be solved simultaneously and we arrive at 

T - 

2 AxM (129) 



CM = ^n 



Aj(u) + AyoM 



CM - -? 



-2^vo(^) (130) 



yyjjj — t 2 j. 2 



i ze 2 ( w ) + i vo 2 (^) • 
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